Modeling with dependent failures

My broad research interest is in dependable systems, in particular developing fault-tolerant distributed algorithms and applying them to practical problems. Developing dependable systems is an important goal as we increasingly rely upon large-scale wide-area distributed systems to support a wide range of online services. As systems scale in size and extent, efficiently coping with failures is an essential requirement for providing highly-available services. Tolerating failures efficiently depends upon the type and frequency of failures in the system, and the failure model assumed to account for them. A typical assumption when designing dependable systems is that failures are independent. This assumption, however, is increasingly unrealistic for emerging distributed systems because individual events can cause extensive dependent failures. For example, faults in shared infrastructure, such as power and cooling in machine rooms and hosting centers, can incapacitate multiple racks of machines at a time. Faults in shared resources, such as physical network connections and exchanges, can introduce severe network partitions. Large-scale Internet attacks, such as viruses and worms, can lead to wide-spread exploitation and failure of hundreds of thousands of hosts. Given the prevalence and severity of dependent failures, developing methods for designing systems that explicitly model dependent failures has the potential to improve their efficiency and reliability. By explicitly modeling dependent failures, distributed systems can improve their efficiency by using fewer replicas or reducing their communication latency and overhead. They can improve, for instance, availability for the same number of replicas compared to replica selection assuming independent failures. When designing algorithms, a common conservative assumption is that it is possible to determine a threshold t on the number of process failures, where processes abstract units of computation in the system. This assumption is conservative because distinct subsets of processes of the same size might have different failure probabilities, and the value of the threshold must be the size of the largest subset that achieves a desired level of availability. Using this “threshold model” is adequate when failures are independent and identically distributed: all subsets of size t have the same failure probability. However, if this assumption does not hold and we still use a threshold, then we might be missing the opportunity of using fewer replicas. This observation leads to the following questions: